A Comparison of Usability Assessment Methods Applied to the U.S. Navy’s Performance Management and Appraisal System

Usability testing is an important method for examining how system users understand and use a system to complete a specific task. This research method is used by system developers to test and evaluate automated tools ranging from information systems, web-based questionnaires, and websites. RTI International recently implemented a large-scale, multi-site usability testing effort to assist in the development of the U.S. Navy’s new web-based performance management system. As part of this testing effort, several methodological variations of usability testing were applied. This paper describes the relative utility of these usability methods and examines the differences that resulted from each. The data presented in this paper include behavior coding of usability tests, timing estimates from usability tests, and subjective data from preand post-test user surveys and focus group results.